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A study of ways in which users conduct subject 
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students. The OPAC used is menu-driven, and provides access through 
author, title, combined author/title, subject (LC subject headings), 
keyword, and number; boolean searching was not fully implemented at 
the time of the study. Each subject was asked to complete two search 
tasks, one straightforward and easy to complete, and the other 
open-ended and more difficult. They were then asked to list the call 
numbers of relevant items and to fill out a questionnaire designed to 
ascertain user satisfaction and demographic information. Results were 
considered for three criterion measures: user satisfaction, number of 
hits, and relevancy score. It was found that (1) the subjects used 
the OFAC with relative ease and the degree of success and 
satisfaction obtained was relatively high; (2) most subjects 
preferred subject heading to keyword searching, and (3) there was no 
evidence of strong relationships between search type or search 
results and individual characteristics. (BBM) 
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Searching the Online Public. Access Catalog 

Online public access catalogs <OPAC) offer opportunities and 
challenges for both librarians and patrons. An OPAC offers 
patrons the potential for a multitude of access points to the 
col lection and the problems dealing with complex, interactive 
systems. CPACs increase bibliographic control and relieve 
librarians from the burdens of catalog card production and 
filing, but introduce requirements for managing the interactions 
of people and complex systems. Increased flexibility inevitably 
brings increased complexity for "all concerned. One thi~c is 
certain; OPACs are being developed and installed in increasing 
numbers in libraries of all types. 

Much has been written about the technical aspects of OPAC 
development, some evaluative studies have been done, and propo- 
nents and opponents have argued their cases. That patrons and 
librarians have distinct views of the catalog in general is not a 
new idea and calls for broader access for patrons have been made 
(,<ocke & Ross, 1985) . Not until the Council on Library 

Resources sponsored a massive Online Catalog Project, however, 
were users and systems studied carefully. Five agencies 

(J. Matthews and Associates, Library of Congress, OCLC, Research 
Libraries Group, and University of California) conducted studies 
of thirty-one OPACs under the auspices of the Council. Each 
agency used a common questionnaire but developed individual 
research hypotheses and methods. 
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One aspect of the OCLC project explored user search patterns via 
transaction log analysis. Kaske and Sanders (1983) reported that 

"In all types of libraries, patrons arej conducting more subject 
searches than most librarians generally believe they do (p. 65)-" 

Other OCLC results (Markey, 1983; Tolle, 1983) ir.dicated that 
users ten to use the same search pattern throughout a session, 
e.g. they stick with an author search rather than change to a 
subject search; different users use a variety of search strat- 
egies successfully; that patrons in general are successful in 
locating documents, although subject searches are least success- 
ful; and that patrons have positive attitudes toward OPACs. 
Although much was learned through these explorations of 0!-'ACs, 
all project directors point out that much work remains to be done 
in developing both theoretical and practical understanding of the 
how humans and machines interact, in libraries. 

Purpose The purpose of this research was to explore the ways that 
users conduct subject searches with an online catalog. It was 
conducted to contribute to understanding of how people approach 
search tasks in a controlled environment, to give guidance to 
"designeFs" of DP AC 5, and to provide an experience base for the 
design and evaluation of OPAC training methods and materials. 
Both process (search patterns) and product (search results) 
variables were examined with respect to individual character- 
istics of subjects. User searc* patterns were also related to 



2 



4 



the user/system interface. Since the research was e*pl oratory, 
no a priori hypotheses about relationships were made. 

METHOD 

Variables The interaction «f human beings with complex systems 
presents an array o-f possible variables, many of which are 
individually uncontrollable or interact in nonsummati ve ways. In 
presenting a conceptual framework for general online retrieval, 
Fidel ?< Soergi?l ( 19B3) provided an outline of research variables 
for study which included: setting, user, request, database, 
search system, searcher, search process, and search outcome. For 
the case of an OPAC the user and searcher are one and the same 
as are database and search system. Lawrence & Matthews (1984) 
considered four major ' variables in their study of online 
catalogs: user, task, set i : g, and system interface. 

The present study considered the user, the search process, the 
results, and some aspects of the user interface as major var- 
iables. Setting and the search task (request) were controlled 
by the researchers and thus treated as constants. Characteris- 
tics taken as d: me'S2 ons of the user variable were: previous 
computer experience, previous OPAC experience, university 
status, major, sex, age, and native language. Analysis of 
individual searches led to a. simple binar/ classification scheme 
for search strategy: homogeneous search (e.g. all subject heading 



search types), and heterogeneous search (e.g. a subject heading 
search followed by a keyword search). Results were considered 
for three criterion measures: user satisfaction, number of hits, 
and relevancy score. The user interface was considered by 
examining frequencies of command use. 

- iyb igcts Volunteers were solicited from computer science, 
library science, education, and psychology classes. A total of 
39 volunteers participated in the study. Table 1 summarizes 
subject characteristics. Overall, subjects were experienced 
computer users (74*/. used computers at least weekly). This was an 
considered to be an unusually high proportion of computer 
users (Kaske & Banders (1983) reported that about 257. of the 
users in their study regularly used other computer systems) l and 
was likely due to the type of classes from which the subjects 
were drawn. Over half <547.) had some previous OPAC experience. 
It was highly unlikely that this experience was with the OPAC in 
this study because ct that time it was only available in the Law 
Library at the Baltimore campus. Although the intention of the 
research was to attract a large, representative sample of 
undergraduate and graduate students from a variety of disci- 
plines, the actual number of volunteers was disappointingly low 
and the ranges of statuses and majors was skewed. Most of the 
subjects wer*v Master's level {64'/.) Library Science <627.) stu- 
dents. The intention of the research design was to compare 
search patterns and results across majors. Since so few 
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Table 1 



Subject Characteristics 

Previous computer experience 

daily 14 

weekly IS 

monthly 4 

quarterly 1 

once a year 1 

never 4 
Any previous OPAC ex per. -nee 

yes 20 

no 17 
University status 

fresh man /sophomore i 

junior/senior 8 

Master's 25 

Doctoral 1 

faculty 3 

staff 1 

Sex 

male 8 

female 31 

Major 

Library Science 24 

other 15 

Age 

20-25 14 

26-30 8 

31-35 6 

36-40 4 

41+ 4 
Language 

English 36 

non-Engl x sh 3 

Ngtg: some characteristic totals not 39 due to missing 
responses. 



non-library science subjects were obtained <3 computer science, 4 
education, 4 business, 4 miscellaneous), these were combined and 
considered as a non-library 'science group. The subjects were 
predominately female (BOX) and over half <617.) of all subjects 
were aged 30 years or younger. 



§g££iQfl The study was conducted at the .University of Maryland 
College Park campus . The University of Maryland has -five 
campuses which at the time of the study were in the third year of 
a seven year library automation project. At College Park, the 
collections are housed in a main library and six branches and 
serve approximately 35,000 students and over 10,000 faculty and 
staff. A Beac integrated library system was installed at the 
College Park campus in 1982 to serve the branches of all 
libraries on all five campuses. The circulation subsystem was in 
fuil operation at all campuses before the study began. The 0PAC 
subsystem came online at the Law Library at the Baltimore campus 
in December of 1983. At the time of the study, the College Park 
holdings were being added to the database, and the subsystem was 
not yet available to the public. Approximately 1.2 million items 
(450,000 titles) were represented in the database at the time of 
the study. 

System The DPAC used in this study is menu-driven and gives users 
access through author, title, combined author /title, subject (LC 
subject headings), keyword (keyword in author, title or subject 
heading), and number (call, government document, LC, ISN) . 
Although boolean searching is part of the system design, it was 
not fully implemented during the time the study took place. The 
system first prompts the user for a search type, e.g. subject; 
author, etc. A search specification is then requested, e.g. for 
subject search a subject heading ix requested and one example 
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given. During the search session three character command 
abbreviations or number choices Her titles or subject heading 
lists) are presented as options for the user. Users must type 
the thre« characters or press a function key to select an 
option. The various commands ares FDR (scroll forwards in a 
list), BAC (scroll backwards in a list), IND (retut n to a list of < 
subject headings), CIT (return to a list o< titles), FUL (see a 
full citation), BRF (see shelf location), CAT (begin anew 
search) and HLP (view a help screen).. 

Search Tasks Many studies of both traditional and online 

catalogs have demonstrated high user success for searching the 
catalog (Haf ter, 1979). Subject searching, however, does cause 
some difficulty for users (Bates, 1977; Kaske & Sanders, .1983; 
Kinney, 1984). Since one of the greatest potentials for increas- 
ed service to the patron offered by an OPAC is the provision of 
many subject access points, and as noted earlier, DPAC users 
conduct more subject searches than previous studies have shown, 
this study focused on subject searches. Several search scenarios 
were developed early in the project. DPAC searches were conduct- 
ed by project staff for each, and two searches selected for use 
in the project (Appendix A). The first search task was designed 
to be straightforward and easy to complete. It required a simple 
topic (dog) to be related to a well-defined task (training). The 
second search task *as designed to be open-ended and more 



difficult to complete. It required two broad topics (children 
and television) to be related in every general way (effects of). 

Pr Q£ e^yrss Subjects were told they would have a chance to help 
library staff evaluate the OPrtC which was due to become available- 
on campus. They signed up for a half -hour block of time and were 
told to report to a room in the main campus library. A research 
assistant following a written protocol (Appendix B) presented 
them with a page describing two search scenarios and directed 
them to the terminal. Subjects were asked to list the call 
numbers of items they would actually <=?ek in the stacks. No 
instruction for using the terminal or the system was given. Upon 
completing the search, or when 20 minutes were up, the research 
assistant gave the subject the questionnaire to complete. The 
twenty minute period was based on pilot tests of the searches by 
graduate assistants and the OCLC results (Tolle, 19B3) which 
yielded £.n average search timf.s 6.1 minutes at Syracuse 
University and a range of four to nine minutes per search at Ohio 
State University (both of which were similar settings to the one 
in this study). The data collection took place over an eight week 
period. Each week, a research assistant ran a program to dump 
the transaction log file for each subject. 

Criterien_Mgssyres A questionnaire (Appendix C) designed to 
ascertain user satisfaction and demographic information was 
designed based upon the questionnaire used in the CLR Online 
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Catalog Project (Matthews & Lawrence, 1984). This instrument was 
tested with graduate assistants in the Library School and 

revised. Responses to five questions ( i-5) were used to assign a 

» 

user satisfaction rating to each user. The remainder of the, 
questionnaire requested demographic data and comments. The 
sec. or, d, and clearly the grossest measure of performance consist- 
eo of simple counts of "hits" for each search. Subjects selected 
relevant documents Chits) by listing the call numbers on the 
search task forms. The third criterion measure used was a 
relevancy score assigned by the researchers to each search task 
completed. Upon completion of the data collection, titles for 
all call numbers .listed by subjects were retrieved from the 
database and listed. Four project staff members independently 
assigned relevancy ratings to each title using a five point scale 
(Saracevic, 1976 recommended a ratio scale having from two to ten 
points). These ratings were averaged, and a final relevancy 
score assigned to each title. Each subject's set of call numbers 
was matched with the relevancy scores for each call number they 
& listed. A mean relevancy score for each of the two search tasks 
was thus found for each subject. 

Que§li2Q5 Two general questions motivated the research . How are 
subject searches conducted using an OPAC? How successful 
are subject searches using an OPAC? Particular questions which 
guided the study related to the relationships between the major 
variables. Correlations were calculated using the SPSS program 
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to explore the following questions! Are results related to 
individual characteristics? Are search strategies related to 
individual characteristics? Arje results related to how the 
searches were conducted? 

RESULTS . 

* * 

Sgarch_Eatt§_^s^ Complete transaction summaries ^ere col lected 
•for 33 subjects. Transaction logs for six subjects were lost due 
to equipment problems. Since each transaction log file contained 
all system responses as well as the keystrokes entered, an 
abbreviated summary of each subject's session w^s preparr j. Each 
keystroke action was termed a "move" for the purpose of this 
study. Each subject* s session was divided into two parts, one 
each for the two search tasks. These records contained a 
sequence of search beginning types <e. g« subject, subject, 
keyword-title, etc.) and summaries of the total number of 
keystrokes for each of the following moves: forwards, backwards, 
see subject heading lists, mee title lists, see full citations, 
see shelf locations, and helps. Total number of hits listed on 
the search forms came next, followed by the same summary for the 
second search. Table 2 summarizes the search moves -for both 
search tasks. 
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Table 2 



Summary of Search Moves 



, Search Task 1 
Btfl B*Dfi£ 



Search 
flean 



Task 2 
Range 



n egin search 
Forwards 
Backwards 
Subject list 
Title list 
Full citation 
Location 
Help 



3.9 
5.6 
0.9 
1.8 
2.0 
0.8 
0.5 
0.2 



1-14 

0^22 

0-5 

0-8 

0-11 

0-3 

0-3 

0-1 



3.8 
8.6 
2. 1 
2.2 
1.7 
1. 1 
0.5 
0.2 



1-15 

0*~ wO 

0-28 

0-8 

0-11 

0-5 

0-3 

0-3 



It was somewhat surprising that subjects in general did not use. 
more begin search moves for search task two which was more 
open-ended than search task one. Perhaps the substantial number 
of subject headings and title lists for the topic provided enough 
information to satisfy the perceived task. Another reason 
may have been that half of the subjects were stopped after 20 



minutes of searching and thus may not have completed their sec c id 
search. Because there were so many subject headings and titles 



for the second search task, it is not surprising that subjects 
moved forward and backwards significantly more times in the 



second search. Uses of forwards and backwards displays may have 
served as important filtering aids for selecting relevant titles 
since the mean number of hits listed by subjects- for the second 
search task was close to the mean for the first task which had 
many fewer possible titles and subject headings from which to 
select. Use of the subject listings and title listings features 
was not great for either search. For the second search there was 
a slight increase in the generally meager use of full citations. 
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This may have been due to a learning effect for the second 
search, but since other moves did not show a learning effect from 
first to second search, it was more likely due to subjects 
wanting to gather more information about a more difficult topic 
to weed a large collection of hits. That subjects would seldom 
check for shelf location is not surprising considering that they 
were searching for topics that they themselves did not identify. 
Subjects' neglect of the help feature could be interpreted as an 
indication of . how easy the system was to use or an indication of 
t poor help facility. Since only eight subjects tried the help 
feature at any time,, the former interpretation seems most 
plausible. Although very few subject? used the help facility, 
some subjects did make comments on the questionnaire about 
providing command summaries or subject heading lists available. 
See fiarkey (1984), for recommendations for online and offline 
assistance with OPACs. The high degree of success in using the 
OPAC in this project may have been due to the high degree of 
previous computer experience or previous other OPAC experience 
characteristic of this sample. 

Because the OPAC used in the study gave users subject <LC 
subject heading) and keyword (author, title or subject heading) 
access points, subjects' preferences were of interest. Table 3 
summarizes all search types for both search tasks. 
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Table 3 



Summary of Begin Search Moves by Typ 



e of Search 



Search Task 1 



Search Tack 2 



§£*r£b_Iy.Ee. 




Subject heading 
Keyword (total ) 



subject 
Author 
Title 

Combined author /title 
Number 



title 



author 



73 
29 
0 
S 
It 
1 
9 
0 
3 



637. 

25% 
0% 
7% 

18% 
IX 
8'/. 

o% 

3% 



80 
16 

0 




80% 
16% 
0% 
2% 
14% 
0% 
3% 
1% 
0% 



Total 



115 



100 



Subject heading searches were most commonly used for both search 
tasks, but were used even more exclusively in the second task. 
This may have been due to the search topic itself or to a 
learning effect for the second task. Not-? that subjects were 
generally willing to use library assigned subject headings rather 
than their own keywords. It should be noted that the sample 
consisted of mostly library science majors (21 of 33=64'/.). The 
subjects who were non-library science majors generally used a 
variety of search types and were more likely to use keyword 
searches. The 12 (36%) non-library science majors used 48% of 
the keyword searches for the first search task and 88% of all 
keyword searches for the second search task. Beneral i zing from a 
small, non-randomized sample is not possible, but it is clear 
that both subject and keyword searches are used by patrons and 
comparative study across user catagories should be undertaken. 
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As in previous OPAC studies, great variability was shown in how 
subjects defined their searches- Note that subjects had no 
opportunity to plan a search before arriving at the terminal. It 
is likely that some subjects explored a new system rather than 
■focused on execution of an optimal search- Searches which used 
a single search type for a task (e.g. all subject headings) were 
termed homogeneous searches. Searches employing different search 
types for a task were termed heterogeneous searches. Table 4 
preser, Us a summary of how many subjects exhibited homogeneous and 
heterogeneous search patterns. 

Table 4 

Summary of Search Pattern Types 

Search Task 1 Search Task 2 

Pattern.I^ee N % N % 

Homogeneous (total) 20 61% 24 73V. 

subject heading 17 52'/. 19 5B7. 

keywor i 3 97. 5 157. 

other 0 07. 0 07. 

Heterogeneous 13 39% 9 277. 

Most subjects selected a search type (usually subject heading) 

and stayed with it for the search task. Some subjects used a 

single search type one time only, others used a single search 

type repeatedly, and still others used a multiplicity of search 

types. These results are in close agreement with the results 

reported in the OCLC studies^ Tolle (1983) reported that 

overall, two-thirds of the users used a single search type. 
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To explore whether subjects were consistent in search patterns 
across search tasks, a simple comparison between search tasks 
one and two was made. Eleven of the 33 (33%) subjects changed 
their search approach -for the second search task. Thus, 
two-thirds of the subject* were consistent in using a search 
pattern for both tasks. Since; subjects were generally successful 
with the first search it is not surprising that they maintained 
the search approach even for a more difficult search. 

a 

§e§rct3„sy££g§s User Satisfactions Results of the searches were 
generally successful. Scores were -computed for user satisfaction 
by assigning a value <l«most satisfied, — 4-1 east satisfied) to 
each of the four possible responses on the five questions on the 
questionnaire. The mean of the five values was used as a measure 
of user satisfaction. The overall mean was 1.6, a generally high 
level of satisfaction. The data were collapsed in this manner to 
provide an overall measure of satisfaction. Almost all subjects 
made comments, and many provided insight into some of the 
problems subjects encountered. Several tried to use Boolean 
connectives based upon experience with other systems and this led 
to error messages and misleading displays. These frustrations 
were expressed in comments and the satisfaction scores in a few 

Number of Hits: Subjects were asked to copy the call numbers of 
titles which they would actually try to find in the stacks. 



Since subjects were volunteers in an experiment rather than 
patrons motivated by personal needs, interpretations of success 

based on "what if" . relevancy were made cautiously and 

i 

in-formal ly. For the -first search task the mean number of hits 
was 5.3 and the number of hits, identified ranged -from one to 
twelve. For a simple search task with relatively few titles in 
the database, this seemed reasonable. For search task two the 
mean number of "hits was 5.9. As expected, the range of hits 
cited was much greater, ranging from zero to thirty-four . The 
subject who identified no hits made thirteen attempts .to link 
subject headings with boolean operators, yielding repeated error 
messages from the system and no titles. Perhaps subjects were 
reflecting actual users' habit of using the catalog to find a 
relevant shelf section and then browsing the shelves. 

4 

Relevancy score: The mean relevancy score for search task one was 
4.1 based on a 1 to 5 scale with 1 representing -not relevant and 
5 representing highly relevant. Scores ranged from 1.3 to 5.0. 
For the second search task the mean score was 3.8 with a range 
from 1.0 to 4.9. The fact than no subject scored a perfect 5 for 
the second , task reflects the relative open-endedness of the 
second search task. In general, subjects found items that were 
judged by the research team to be quite relevant for both search 
tasks. Overall, the QPAC provides a viable method for locating 
useful documents in the 1 ibr ary. 

16 
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Results and search patterns No strong relationships between 

results and search oattern were -found. Although it seems 
plausible that use of multiple search types will yield more hits 
or more relevant hits, or even more satisfaction with the overall 
search, no such results were found. Table 5 presents Spearman 
correlation coefficients for the three result measures by the twp 
search pattern types- Note that Spearman coefficients were used 
here and in all cases where one or both of <the variables were not 
measured on interval scales. 

Table 5 

Correlation Coefficients for Results by Search Pattern Type 

Search Pattern Type 

Search Task 1 Search Task 2 

Cr i tor i,gn_Mea5yr& BE BE 

Satisfaction -179 .161 .179 .161 

Hits -.039 .416 -.045 .403 

Relevancy score .-.028 .439 -.004 .492 

Notes Satisfaction was measured for the entire session rather 
than each search task. 

Subjects were generally successful in locating items using the 
catalog and satisfied with using it regardless of the search 
approach taken. 

Individual characteristics and search patterns. To explore 

possible relationships between individual characteristics and 
search patterns, correlations were calculated. Table 6 presents 
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Spearman coefficients and probability levels if or the two search 

pattern types by characteristic. 

Table 6 

Correlation Coefficients 
for Individual Characteristics by Search Pattern Type 







Pattern 


Type 






Search 


Task 1 


Search 


Task ! 


Characteristic. 


B 


B 


R 


B 


Previous computer experience 


.056 


.379 


-. 13C 


.236 


Previous OPAC experience 


.017 


.464 


-.091 


.314 


University status - 


.402 


.010 


-.256 


.075 


Se>: 


± 23 


.248 


130 


.236 


Major 


.422 


.007 


. 103 


.284 


Age 


.236 


. 101 


-.087 


.321 


Language 


. 177 


. 163 


.043 


.406 



A somewhat strong relationship was found between university 
status and search pattern type. In general, the larger the 
status number < increasing level of study /work), the lower the 
search pattern type (homogeneous type). In general, the more 
university experience, the more likely a subject was to use a 
direct, single search pattern. Although this relationship was 
found for both search tasks, the findings are tempered by the 
skewed distribution of statuses, where two-thirds (22 of 33) of 
the subjects were Master's level. The strong relationship 
between major and search pattern type for search task one carried 
over only weakly for search task two. Together with the tendency 
of non-library science majors to use a variety of search types, 
this finding provides a basis for further study of academic 
course of study and «. OPAC use. In general however, strong, 
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consistent relationships between search pattern types and 
individual characteristics were not found. 



Ingivigyal characteristics and seargh results R^i * tionships 

between individual characteristics and search results were also 
studied via correlational analyses. Table 7 presents Spearman 
coefficients for individual characteristics by user satisfaction* 

Table 7 

Correlation Coefficients 
for Individual Characteristics by User Satisfaction 

Satisfaction 

Characteristic R g 

Previous computer experience .035 .416 

Previous OPAC experience -.129 .224 

University status .150 « ISO 

Sex -.014 .466 

Major ~. 168 * 153 

Age . 119 .245 

Language .052 .377 

No strong relationships were -found between any individual 
characters ,tics and user satisfaction. The generally high 
overall satisfaction indicated that most users were satisfied 
with the OPAC no matter what the.r personal characteristics. 

Similar results were found for relationships between individual 
characteristics and number of hits. These results are summarized 
in Table 8 below. 
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Table 8 



Correlation Coef f icients 
for Individual Characteristics by Hits 



Characteristic. 



Task 1 Hits 
6 E 



Task 
R 



Hits 
D 



' Previous computer experience 
Previous OPAC experience 
University status 

Se>: 

Major 

Age 

Language 



053 
170 
131 
288 
010 
088 
000 



.376 
. 160 
.216 
.040 
.476 
.307 
.5O0 



181 
-.022 



.042 
129 
-. 195 
166 



220 



. 139 
.450 
.092 
.402 
.221 
. 131 
. 160 



Although no strong, consistent relationships were found, it is 

■ * 

interesting to note the consistent trends in previous experience 
with computers and other OPACs. The consistently negative 
correlations across both tasks indicate that lower experience 
scores, i.e. more actual experience since daily use has the 
lowest value (1) for computer use, and yes to previous OPAC 
experience < 1) relate wix.h high numbers of hits; subjects with 
more computer experience or OPAC experience tended to locate more 
hits. This trend, however , was not found when relevancy sec -es 
were considered. Table 9 presents Spearman coefficients for 
individual characteristics by relevancy score. No strong, 
consistent relationships between relevancy score and individual 
characteristics were found. 
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Table 9 



Correlation Coefficients 






for Individual Character i sti cs by 


Relevancy 


CJ#- pttr- QC 






Task 1 Score 


Task 2 


Score 




R 


e 


R 


B 


Previous computer experience 


. 100 


.275 


-.429 


.005 


Previous OPAC experience 


-.097 


.287 


.072 


.342 


University status 


. 126 


.226 


.034 


.423 


Sex 


. 154 


. 178 


. 171 


. 159 


Major 


-.017 


.459 


-.250 


.070 


Age 


. 124 


.239 


-. 109 


.269 


Language 


-.221 


.091 


-.005 


.489 



DISCUSSION AND SUMMARY 
Perhaps the major finding of this research was the relative ease 
with which subjects used the OPAC and relative high degree of 
success and satisfaction obtained. In general, subjects prefer- 
red subject heading searches rather than keyword searches, but 
used a variety of approaches nonetheless. Subjects used the 
commands for specifying a search type and moving forwards in a 
list most often, and seemed to ignore the help feature. Neither 
of the major independent variables, search pattern type or search 
results were related to any of the individual characteristics at 
statistically significant levels. Major field of study and 
university status were weakly related to search pattern type, and 
previous computer and OPAC experience were weakly related to 
number of hits. These relationships bear further study with a 
randomly selected, comprehensive sample. It is recommended that 
future studies which control the search task variable should 
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pay subjects -for participation. This should insure at least a 
larger sample,, although not a random one. The -fact that most 
subjects Mere library science students* with intrinsic interest in 
OPACs, together with the fact that the OPAC was not yet in . 
operation (novelty effect) likely caused them to focus on the 
tool rather than the tasks at hand. The long search times for 1 
subjects in this project may have been due to their willingness 
to explore the system, thus affecting the resulting search 
patterns. 

Based on subjects* comments, it is recommended that command 
summaries and simple system overviews be posted near terminals. 
Although some subjects suggested immediate access to subject 
headings or thesauri in print or online form, whether these are 

cost effective remains an issue. A few subjects recommend 

j 

making print versions of hits available. Por this sample, 
training in the use of the system was not necessary or request- 
ed. Further exploration of users training requirements using a 
more representative sample i s recommended . Overall, the results 
of this project suggest that this OPAC will be well received and 
used in a fashion consistent with previously studied systems. 
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Appendix A 
Search Task© 
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ONLINE CATALOG SUBJECT SEARCHES 



Conduct a March of the Online Catalog to locate books which 
will help you acquire information relevant to tha following 
topica. Xr tha apaca balow each topic write tha call numbers 
of the tilt.es you would take tha time to find on the' 
shelves. 0 



Sestcb I* You have Just gotten a dog. 
finding out how to train your new pet. 



Ysu are interested in 



56dr.£h 2- You are writing a paper related to the social 
impact of television oh children. You need background 
information on this topic. 



u 
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Appendix B 
Protocols 
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OPAL PROTOCOLS 



1. IcLl lyfiitStt* are exploring the online catalog which 
will become avail able* to the Co! leg* Park campus in the near 
future. By conducting two sample searches for us, *nd giving 
us feedback on this activity you will assist us in evaluating 
the system* and preparing for its general introduction. 

a 

Here are the subjects for which you are to locate books, give. 

tbc cubicst tbc ie*r.cbci and 2 biqbH.- 

You will be searching the computerized card catalog with this 
terminal. To signal the computer that you are done typing a 
command, press the SEND key. <BfiiQt tfi tbC 1C0S* fcfiY.> I will 
be unable to answer questions! once you begin. As you locate 
books that you would take the time to find on the shelves, 
please Jot down the call numbers under the topic on the 
paper. When you have completed the searches, I will give you 
a short questionnaire to complete. 

Be met tbsfc tbc lxitciD if ccctfxx tbc ocio stou ibsuld fee so 
tbc iccttox Cecil Cfil cob" nod Is set tbc ycec itcrteg- 
Beit tbc tice tbc iciiiso ii ifccctiDQ so tbe bbbiqo. ibtet- 

if tbc mfeicEt ti&ei ost SBSBicts tbc uiccbti io 20 oinytEs*. 
isE ibis ts its& cod SB&slctc tbc flyeitiflnoiice cdy*!!** 

Collect tbc iffitcb ibeet fees tbi WBisci- 

Nste tbc tise tbc mbicst cfis&ietcfil tbc ifiexsbes By. ssYiog 

C«Si< tfi tbB ffiiiD ffiCQU liUUf tb* ICC EBflMDsDSlij. §ivg the 

lUBiest tbc flyeitiBQOiice*. iiYiogi 

Please complete this questionnaire. Do not put your name or 
any identifying marks on it. 

while tbc iybic£t ii CMEltiiDfl tbc guti&i&QOiiccx cbecE tbs 

eBBCBBCiitC CIIEBQICC f SZ Y.BUC ififirfSfignt fi£ the. sybigEt SO 

tbt Esdios ibeet* 

CelUct tbc gycitifiopiicc too* BCQBil*. fiiy.e tbc lyteicct 2 

BUBSbUCC dClCtibiDQ tbC CYltCiDi. Se¥* Thank *ou for 
participating in this study. Here is a brief description of 
the system to be implemented. 

SfcsBlc tbc BBdiofi ibect iaz tbc lyfeicct*. tbs flyettisoosicSi. 

tnti* tbC ICitCb f BCffl tfiSCtOCr.*. iOd BCCfiare £gr tbt next 

lybiCBti. 
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CODING SHEET 

Data i . 

Starting Timet 

Ending Tinea 

Did the subject finish both searches? 

Your impression of the subject* si 



extremely extremely 

high high average low low 

Level of Interest 12 3 4 5 

Level of Patience 12 3 4 5 

Level of Seriousness 12 3 4 5 



Comments: 
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Appendix C 
Subject Guest i onnaire 



ONLINE CATALOG QUESTIONNAIRE 



We trt exploring the online catalog which will become available to 
the College Park Campus in the near -future. Please help us by 
completing the following questions. Your responses are 
confidential, please do not put your name anywhere on the 
questionnaire. Circle the letter of you response (s) . Thank you. 

1. In this computer search I found (mark one only): 

a. more than I was looking for 

b. all that I was looking for 

c. some of what I was looking for 

d. nothing I was looking for 

2. In relation to what I was looking for, this search was <mark one 
only) : 

a. very satisfactory 

b. some hat satisfactory 

c. somewhat unsatisfactory 

d. very unsatisfactory 

3. In terms of ease of use, the online catalog is (mark one only) s 

a. very easy to use 

b. somewhat easy to use 

c. somewhat difficult to use 

d. very difficult to use 

4. My overall attitude toward the online catalog is (mark one only) s 

a* very favorable 

b. somewaht favorable 

c. somewhat unfavorable 

d. very unfavorable 

5. Compared to the card catalog in this library, the online catale* 
is (mark one onl y> t 

a„ better 

b. about the same 

c. worse 

d. can't decide 

6. Which search types did you like (mark all that apply)? 

a. subject heading 

b . sub j ect keyword 

c. title keyword 

d. title 

e. author 

f. other 



7. I use some type(s) of computer (s> t 

a. daily 

b. weekly 

c. monthly 

d. about four times a year 

e. about once a year 

f . never 
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B, Have you ever used any other online catalog? 
a • yes 
b • no 

9. My present affiliation with this university is: 

a. f r eshman / sop homo r e 

b . j un i or /sen i or 

c. graduate- Masters level 

d. graduate- doctoral level 

e. faculty 

f. staff 

g. other 

10. X am: 

a. male 
b • female 

SHORT ANSWER. Please write your response below each question. 

11. My major or area of study iss 

12. How old are you? 

13. Is English your native language? 

14. Was there anything confusing ^bout the online catalog? What? 



17. Do you have questions, comments or suggestions about using the 
online catalog? 



THANK YOU FOR YOUR COOPERATION 
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